Distributed learning and inference in deep models

In recent years, the size of deep learning problems has been increased significantly, both in terms of the number of available training samples as well as the number of parameters and complexity of the model. In this thesis, we considered the challenges encountered in training and inference of large deep models, especially on nodes with limited computational power and capacity. We studied two classes of related problems; 1) distributed training of deep models, and 2) compression and restructuring of deep models for efficient distributed and parallel execution to reduce inference times. Especially, we considered the communication bottleneck in distributed training and inference of deep models. Data compression is a viable tool to mitigate the communication bottleneck in distributed deep learning. However, the existing methods suffer from a few drawbacks, such as the increased variance of stochastic gradients (SG), slower convergence rate, or added bias to SG. In my Ph.D. research, we have addressed these challenges from three different perspectives: 1) Information Theory and the CEO Problem, 2) Indirect SG compression via Matrix Factorization, and 3) Quantized Compressive Sampling. We showed, both theoretically and via simulations, that our proposed methods can achieve smaller MSE than other unbiased compression methods with fewer communication bit-rates, resulting in superior convergence rates. Next, we considered federated learning over wireless multiple access channels (MAC). Efficient communication requires the communication algorithm to satisfy the constraints imposed by the nodes in the network and the communication medium. To satisfy these constraints and take advantage of the over-the-air computation inherent in MAC, we proposed a framework based on random linear coding and developed efficient power management and channel usage techniques to manage the trade-offs between power consumption and communication bit-rate. In the second part of this thesis, we considered the distributed parallel implementation of an already-trained deep model on multiple workers. Since latency due to the synchronization and data transfer among workers adversely affects the performance of the parallel implementation, it is desirable to have minimum interdependency among parallel sub-models on the workers. To achieve this goal, we developed and analyzed RePurpose, an efficient algorithm to rearrange the neurons in the neural network and partition them (without changing the general topology of the neural network) such that the interdependency among sub-models is minimized under the computations and communications constraints of the workers.Ph.D

A Comparison of Risperidone and Buspirone for Treatment of Behavior Disorders in Children with Phenylketonuria

How to Cite This Article: Fayyazi A, Salari E, Khajeh A, Ghajarpour A. A Comparison of Risperidone and Buspirone for Treatment ofBehavior Disorders in Children with Phenylketonuria. Iran J Child Neurol. 2014 Autumn; 8(4):33-38.AbstractObjectiveMany patients with late-diagnosed phenylketonuria (PKU) suffer from severe behavior problems. This study compares the effects of buspirone and risperidone on reducing behavior disorders in these patients.Materials & MethodsIn this crossover clinical trial study, patients with severe behavior disorders after medical examination were randomly divided into two groups of two 8-week crossover treatments with risperidone or buspirone. Patient behavioral disorders before and after treatment by each drug was rated by parents on the Nisonger Child Behavior Rating Form (NCBRF), and after treatment by each drug, were assessed by a physician through clinical global impression (CGI).ResultsThirteen patients were able to complete the therapy period with these two medications.The most common psychiatric diagnoses were intellectual disability accompanied by pervasive developmental disorder NOS, and intellectual disability accompanied by autistic disorder. Risperidone was significantly effective in reducing the NCBRF subscales of hyperactivity disruptive/ stereotypic, and conduct problems. Treatment by buspirone only significantly decreased the severity of hyperactivity, but other behavior aspects showed no significant differences. Assessment of the severity of behavior disorder after treatment by risperidone and buspirone showed significant differences in reducing hyperactivity and masochistic/stereotype.ConclusionAlthough buspirone is effective in controlling hyperactivity in patients with PKU, it has no preference over risperidone. Therefore, it is recommended as an alternative to risperidone.ReferencesSmith I, Nowles JK. Behaviour in early treated phenylketonuria: a systematic review. Eur J Pediatr 2000;159:89-93.Targum SD and Lang W .Neurobehavioral Problems Associated with Phenylketonuria. Psychiatry (Edgemont) 2009; 7(12):29–32.Luciana M, Hanson K L,Whitley C B.A preliminary report on dopamine system reactivity in PKU: acute effects of haloperidol on neuropsychological, physiological, and neuroendocrine functions. Psychopharmacology 2004;175: 18–25.Pappadopulos E, Woolston S, Chait A, Perkins M, Connor DF, Jensen P S. Pharmacotherapy of Aggression in Children and Adolescents: Efficacy and Effect Size. J CDN ACAD Child Adolesc Psychiatry 2006; 15(1):27-39.Shea S, Turgay A, Carroll A, Schulz M, Orlik H ,Smith I and et al. Risperidone in the Treatment of Disruptive Behavioral Symptoms in Children With Autistic and Other Pervasive Developmental Disorders. Pediatrcs 2004; 114:634-641.Miral S, Gencer O, Inal-Emiroglu F.N, Baykara B, Baykara A, Dirik E. Risperidone versus haloperidol in children and adolescents with AD: a randomized, controlled, doubleblind trial. Eur Child Adolesc Psychiatry 2008; 17:1–8.Aman M.G, Hollway J.A, McDougle C.J, Scahill L, Tierney E, McCracken J.T and et al. Cognitive effects of risperidone in children with autism and irritable behavior. J. Child Adolesc. Psychopharmacol 2008; 18:227–236.Pandina G.J, Bossie C.A, Youssef E, Zhu Y, Dunbar F. Risperidone improves behavioral symptoms in children with autism in a randomized, double-blind, placebocontrolled trial. J Autism Dev Disord 2007; 37:367–373.Luby J, Mrakotsky C, Stalets M.M, Belden A, Heffelfinger A, Williams M and et al. Risperidone in preschool children with autistic spectrum disorders: an investigation of safety and efficacy. J. Child Adolesc. Psychopharmacol 2006; 16:575–587.Nagaraj R, Singhi P, Malhi P. Risperidone in children with autism: randomized, placebo controlled, double blind study. J. Child Neurol 2006; 21:450–455.Haas M, Karcher K, Pandina GJ. Treating Disruptive Behavior Disorders with Risperidone: A 1-Year, Open-Label Safety Study in Children and Adolescents. Journal of Child and Adolescent Psychopharmacology 2008;18(4): 337–346.Jensen P, Buitelaar J, Pandina G, Binder C, Haas M. Management of psychiatric disorders in children and adolescents with atypical antipsychotics. Eur Child Adolesc Psychiatry 2007; 16:104–120.Pandina G, Aman M, Findling R. Risperidone in the management of disruptive behavior disorders. J Child Adolesc Psychopharmacol 2006; 16:379–392.Reyes M, Buitelaar J, Toren P, Augustyns I, Eerdekens M. A randomized, double-blind, placebo-controlled study of risperidone maintenance treatment in children and adolescents with disruptive behavior disorders. Am J Psychiatry 2006; 163:402–410.Reyes M, Croonenberghs J, Augustyns I, Eerdekens M. Long-term use of risperidone in children with disruptive behavior disorders and subaverage intelligence: Efficacy, safety, and tolerability. J Child Adolesc Psychopharmacol 2006; 16:260–272.Croonenberghs J, Fegert JM, Findling RL, De Smedt G, Van Dongen S. Risperidone Disruptive Behavior Study Group: Risperidone in children with disruptive behavior disorders and subaverage intelligence: A 1-year, openlabel study of 504 patients. J Am Acad Child Adolesc Psychiatry 2005; 44:64–72.Aman M G, Smedt G D, Derivan A, Lyons B, Findling R L.Double-Blind, Placebo-Controlled Study of Risperidone for the Treatment of Disruptive Behaviors in Children With Subaverage Intelligence. Am J Psychiatry 2002; 159:1337–1346.Snyder R, Turgay A, Aman M, Binder C, Fisman S, Carroll A. Risperidone Conduct Study Group. Effects of risperidone on conduct and disruptive behavior disorders in children with subaverage IQs. J Am Acad Child Adolesc Psychiatry 2002; 41:1026–1036.Gualtieri T. Buspirone for the Behavior problems of patients with Organic Brain Disorders. J Clin Psycopharmacol 1991; 11:280-281.Stanislav S, Fabre T, Crismon L, Childs A. Buspirone’s Efficacy in Organic-Induced Aggression.J Clin Psycopharmacol 1994;14:126-130.Realmuto GM, August GJ, Garfinkel BD. Clinical effect of buspirone in autistic children. J Clin Psychopharmacol 1989; 9(2):122-5.Ratey J, Sovner R, Parks A, Rogentine K. Buspirone treatment of aggression and anxiety in mentally retarded patients: a multiple-baseline, placebo lead-in study. J Clin Psychiatry 1991; 52(4):159-62.Sorensen MJ, Thomsen PH, Bilenberg N. Parent and child acceptability and staff evaluation of K-SADA-PL: a pilot Study. European Child and Adolescent Psychiatry 2007; 16(5): 293-7.Kaufman J, Brimaher B, Bren, D, Rao U, Flynn C, Moreci P and et al. Schedule for affective disorders and Schizophrenia For school –age children – present and lifetime version- (K-SADS-PL): initial reliability and validity date. Journal of the American Academy of Child and Adolescent Psychiatry 1997; 36(7):980-988. Aman MG, Tassé MJ, Rojahn J, and Hammer D: The Nisonger CBRF: a child behavior rating form for children with developmental disabilities. Research in Developmental Disabilities 1996;17:41–57.Tassé MJ, Aman MG, Hammer D, Rojahn J: The Nisonger Child Behavior Rating Form: age and gender effect and norms. Research in Developmental Disabilities 1996; 17:59–75.Norris M, Lecavalier L. Evaluating the validity of the Nisonger Child Behavior Rating Form – Parent Version. Research in Developmental Disabilities 2011; 32:2894– 2900.Guy W. Clinical global impressions. In: ECDEU Assessment Manual for Psychopharmacology. Rockville, MD, National Institute of Mental Health. 1976.Pp.217- 222.National Institute of Mental Health. CGI (Clinic Global Impression) Scale. Psychopharmacology Bull 1985; 21:839-843.Marder S, Hurford IM, van Kammen DP: Second- Generation Antipsychotics: Biological Therapies. Sadock BJ, Sadock VA, Pedro R: Kaplan & Sadock’s Comprehensive Textbook of Psychiatry, 9th Edition. Philadelphia. Lippincott Williams & Wilkins. 2009. P.3220

Trend in incidence of gastric adenocarcinoma by tumor location from 1969–2004: a study in one referral center in Iran

AIM: In recent years several studies have shown increasing rate of upper gastric cancers regarding to decrease in distal gastric cancers. The aim of this study was to describe the trend of gastric cancers by location in Iran, which is one of the countries with high prevalence of gastric cancers. METHODS: All registered cases of gasterectomy in Tehran Cancer Institute from 1969 through 2004 were re-evaluated clinicopathologically. The stomach was anatomically divided into the upper, middle, and lower third. The prevalence of gastric cancers in 5 year periods estimated by location and the changes trough the time was evaluated independently and in aspect of age and sex. RESULTS: Over 36 years, the prevalence of cancers in the upper and middle third of the stomach have increased and that of the lower third has decreased. These changes were seen in both sexes and age groups under and over 50 and it was more significant in younger. CONCLUSION: The results are the same as most previous reports in other countries. This can indicate different risk factors as well as confrontation with them. However in regard to few numbers of cases in this study, a population-based study is recommended for confirmation

IMAGE SEPARATION BASED ON NONSUBSAMPLED CONTOURLET(NSCT)

ABSTRACT When an image or a document on a paper acquired through scannin

Avaliação Qualitativa da Água Engarrafada e Armazenadas em Frascos de Tereftalato de Polietileno com Base Compostos Químicos Orgânicos

Polyethylene terephtalate (PET) is commonly used for bottling drinking water. PET must be harmless in the sense of the migration potentially unsafe materials into its content. The quality determination of migrated organic chemicals in 15 bottled water stored in PET was performed by gas chromatography-mass spectrometry technique. Most of the organic chemical compounds including phthalate, alkyl phenol, higher alkene and organic acid were detected in the samples. However, no carcinogens and hormones were recognized in the analyzed waters. The most migrated compounds identified between 13 to 100% of bottled water. The findings of present study could be alarming for the food safety legislative establishments in Iran due to the existence of some organic compounds with adverse influence on human wellbeing. Further investigation is recommended to evaluate the risk assessment of the public health arising from the presence of these toxic contaminants in the bottled water consumed by the people.Tereftalato de Polietileno (PET) é comumente usado para engarrafamento de água potável. PET deve ser inofensivo no sentido da migração materiais potencialmente inseguros em seu conteúdo. A determinação da qualidade de produtos químicos orgânicos que migraram em 15 garrafas de água armazenada em PET foi realizada pela técnica de cromatografia gasosa acoplada a espectrometria de massa. A maior parte dos compostos químicos orgânicos, incluindo ftalato, alquil fenol, alceno de maior peso molecular e ácido orgânico foram detectados nas amostras. No entanto, não há agentes cancerígenos e hormônios foram detectados nas águas analisadas. Os compostos migraram identificados entre 13 a 100% de água engarrafada. As conclusões do presente estudo poderia ser alarmante para os estabelecimentos legislativos de segurança alimentar no Irã devido à existência de alguns compostos orgânicos com influência negativa no bem-estar humano. Outras investigações é recomendado para avaliar a avaliação de risco da saúde pública decorrente da presença desses contaminantes tóxicos na água engarrafada consumida pelas pessoas

Global, regional, and national age-sex-specific mortality and life expectancy, 1950-2017 : a systematic analysis for the Global Burden of Disease Study 2017

Background Assessments of age-specific mortality and life expectancy have been done by the UN Population Division, Department of Economics and Social Affairs (UNPOP), the United States Census Bureau, WHO, and as part of previous iterations of the Global Burden of Diseases, Injuries, and Risk Factors Study (GBD). Previous iterations of the GBD used population estimates from UNPOP, which were not derived in a way that was internally consistent with the estimates of the numbers of deaths in the GBD. The present iteration of the GBD, GBD 2017, improves on previous assessments and provides timely estimates of the mortality experience of populations globally. Methods The GBD uses all available data to produce estimates of mortality rates between 1950 and 2017 for 23 age groups, both sexes, and 918 locations, including 195 countries and territories and subnational locations for 16 countries. Data used include vital registration systetns, sample registration systetns, household surveys (complete birth histories, summary birth histories, sibling histories), censuses (summary birth histories, household deaths), and Demographic Surveillance Sites. In total, this analysis used 8259 data sources. Estimates of the probability of death between birth and the age of 5 years and between ages 15 and 60 years are generated and then input into a model life table system to produce complete life tables for all locations and years. Fatal discontinuities and mortality due to HIV/AIDS are analysed separately and then incorporated into the estimation. We analyse the relationship between age-specific mortality and development status using the Socio-demographic Index, a composite measure based on fertility under the age of 25 years, education, and income. There are four main methodological improvements in GBD 2017 compared with GBD 2016: 622 additional data sources have been incorporated; new estimates of population, generated by the GBD study, are used; statistical methods used in different components of the analysis have been further standardised and improved; and the analysis has been extended backwards in time by two decades to start in 1950. Findings Globally, 18.7% (95% uncertainty interval 18.4-19.0) of deaths were registered in 1950 and that proportion has been steadily increasing since, with 58.8% (58.2-59.3) of all deaths being registered in 2015. At the global level, between 1950 and 2017, life expectancy increased from 48.1 years (46.5-49.6) to 70.5 years (70.1-70.8) for men and from 52.9 years (51.7-54.0) to 75.6 years (75.3-75.9) for women. Despite this overall progress, there remains substantial variation in life expectancy at birth in 2017, which ranges from 49.1 years (46.5-51.7) for men in the Central African Republic to 87.6 years (86.9-88.1) among women in Singapore. The greatest progress across age groups was for children younger than 5 years; under-5 mortality dropped from 216.0 deaths (196.3-238.1) per 1000 livebirths in 1950 to 38.9 deaths (35.6-42.83) per 1000 livebirths in 2017, with huge reductions across countries. Nevertheless, there were still 5.4 million (5.2-5.6) deaths among children younger than 5 years in the world in 2017. Progress has been less pronounced and more variable for adults, especially for adult tnales, who had stagnant or increasing mortality rates in several countries. The gap between male and female life expectancy between 1950 and 2017, while relatively stable at the global level, shows distinctive patterns across super-regions and has consistently been the largest in central Europe, eastern Europe, and central Asia, and smallest in south Asia. Performance was also variable across countries and time in observed mortality rates compared with those expected on the basis of development. Interpretation This analysis of age-sex-specific mortality shows that there are remarkably complex patterns in population mortality across countries. The findings of this study highlight global successes, such as the large decline in under-5 mortality, which reflects significant local, national, and global commitment and investment over several decades. However, they also bring attention to mortality patterns that are a cause for concern, particularly among adult men and, to a lesser extent, wotnen, whose mortality rates have stagnated in many countries over the time period of this study, and in some cases are increasing. Copyright C) 2018 The Author(s). Published by Elsevier Ltd.Peer reviewe